3D Gaussian Splatting for Real-Time Radiance Field Rendering

Radiance Field methods have recently revolutionized novel-view synthesis of scenes captured with multiple photos or videos. However, achieving high visual quality still requires neural networks that are costly to train and render, while recent faster methods inevitably trade off speed for quality. For unbounded and complete scenes (rather than isolated objects) and 1080p resolution rendering, no current method can achieve real-time display rates. We introduce three key elements that allow us to achieve state-of-the-art visual quality while maintaining competitive training times and importantly allow high-quality real-time (>= 30 fps) novel-view synthesis at 1080p resolution. First, starting from sparse points produced during camera calibration, we represent the scene with 3D Gaussians that preserve desirable properties of continuous volumetric radiance fields for scene optimization while avoiding unnecessary computation in empty space; Second, we perform interleaved optimization/density control of the 3D Gaussians, notably optimizing anisotropic covariance to achieve an accurate representation of the scene; Third, we develop a fast visibility-aware rendering algorithm that supports anisotropic splatting and both accelerates training and allows realtime rendering. We demonstrate state-of-the-art visual quality and real-time rendering on several established datasets.Comment: https://repo-sam.inria.fr/fungraph/3d-gaussian-splatting

FreeStyleGAN: Free-view Editable Portrait Rendering with the Camera Manifold

International audienceCurrent Generative Adversarial Networks (GANs) produce photorealisticrenderings of portrait images. Embedding real images into the latent spaceof such models enables high-level image editing. While recent methodsprovide considerable semantic control over the (re-)generated images, theycan only generate a limited set of viewpoints and cannot explicitly controlthe camera. Such 3D camera control is required for 3D virtual and mixedreality applications. In our solution, we use a few images of a face to perform3D reconstruction, and we introduce the notion of the GAN camera manifold,the key element allowing us to precisely define the range of images that theGAN can reproduce in a stable manner. We train a small face-specific neuralimplicit representation network to map a captured face to this manifoldand complement it with a warping scheme to obtain free-viewpoint novel-view synthesis. We show how our approach ś due to its precise cameracontrol ś enables the integration of a pre-trained StyleGAN into standard 3Drendering pipelines, allowing e.g., stereo rendering or consistent insertionof faces in synthetic 3D environments. Our solution proposes the first trulyfree-viewpoint rendering of realistic faces at interactive rates, using onlya small number of casual photos as input, while simultaneously allowingsemantic editing capabilities, such as facial expression or lighting changes

End-to-end Sampling Patterns

Sample patterns have many uses in Computer Graphics, ranging from procedural object placement over Monte Carlo image synthesis to non-photorealistic depiction. Their properties such as discrepancy, spectra, anisotropy, or progressiveness have been analyzed extensively. However, designing methods to produce sampling patterns with certain properties can require substantial hand-crafting effort, both in coding, mathematical derivation and compute time. In particular, there is no systematic way to derive the best sampling algorithm for a specific end-task. Tackling this issue, we suggest another level of abstraction: a toolkit to end-to-end optimize over all sampling methods to find the one producing user-prescribed properties such as discrepancy or a spectrum that best fit the end-task. A user simply implements the forward losses and the sampling method is found automatically -- without coding or mathematical derivation -- by making use of back-propagation abilities of modern deep learning frameworks. While this optimization takes long, at deployment time the sampling method is quick to execute as iterated unstructured non-linear filtering using radial basis functions (RBFs) to represent high-dimensional kernels. Several important previous methods are special cases of this approach, which we compare to previous work and demonstrate its usefulness in several typical Computer Graphics applications. Finally, we propose sampling patterns with properties not shown before, such as high-dimensional blue noise with projective properties

ROAM: Robust and Object-aware Motion Generation using Neural Pose Descriptors

Existing automatic approaches for 3D virtual character motion synthesis supporting scene interactions do not generalise well to new objects outside training distributions, even when trained on extensive motion capture datasets with diverse objects and annotated interactions. This paper addresses this limitation and shows that robustness and generalisation to novel scene objects in 3D object-aware character synthesis can be achieved by training a motion model with as few as one reference object. We leverage an implicit feature representation trained on object-only datasets, which encodes an SE(3)-equivariant descriptor field around the object. Given an unseen object and a reference pose-object pair, we optimise for the object-aware pose that is closest in the feature space to the reference pose. Finally, we use l-NSM, i.e., our motion generation model that is trained to seamlessly transition from locomotion to object interaction with the proposed bidirectional pose blending scheme. Through comprehensive numerical comparisons to state-of-the-art methods and in a user study, we demonstrate substantial improvements in 3D virtual character motion and interaction quality and robustness to scenarios with unseen objects. Our project page is available at https://vcai.mpi-inf.mpg.de/projects/ROAM/.Comment: 12 pages, 10 figures; project page: https://vcai.mpi-inf.mpg.de/projects/ROAM

Hybrid Image-based Rendering for Free-view Synthesis

International audienceImage-based rendering (IBR) provides a rich toolset for free-viewpoint navigation in captured scenes. Many methods exist, usually with an emphasis either on image quality or rendering speed. In this paper we identify common IBR artifacts and combine the strengths of different algorithms to strike a good balance in the speed/quality tradeoff. First, we address the problem of visible color seams that arise from blending casually-captured input images by explicitly treating view-dependent effects. Second, we compensate for geometric reconstruction errors by refining per-view information using a novel clustering and filtering approach. Finally, we devise a practical hybrid IBR algorithm, which locally identifies and utilizes the rendering method best suited for an image region while retaining interactive rates. We compare our method against classical and modern (neural) approaches in indoor and outdoor scenes and demonstrate superiority in quality and/or speed

Glossy Probe Reprojection for Interactive Global Illumination

International audienceRecent rendering advances dramatically reduce the cost of global illumination. But even with hardware acceleration, complex light paths with multiple glossy interactions are still expensive; our new algorithm stores these paths in precomputed light probes and reprojects them at runtime to provide interactivity. Combined with traditional light maps for diffuse lighting our approach interactively renders all light paths in static scenes with opaque objects. Naively reprojecting probes with glossy lighting is memory-intensive, requires efficient access to the correctly reflected radiance, and exhibits problems at occlusion boundaries in glossy reflections. Our solution addresses all these issues. To minimize memory, we introduce an adaptive light probe parameterization that allocates increased resolution for shinier surfaces and regions of higher geometric complexity. To efficiently sample glossy paths, our novel gathering algorithm reprojects probe texels in a view-dependent manner using efficient reflection estimation and a fast rasterization-based search. Naive probe reprojection often sharpens glossy reflections at occlusion boundaries, due to changes in parallax. To avoid this, we split the convolution induced by the BRDF into two steps: we precompute probes using a lower material roughness and apply an adaptive bilateral filter at runtime to reproduce the original surface roughness. Combining these elements, our algorithm interactively renders complex scenes while fitting in the memory, bandwidth, and computation constraints of current hardware

Genome-wide association analysis of thirty one production, health, reproduction and body conformation traits in contemporary U.S. Holstein cows

<p>Abstract</p> <p>Background</p> <p>Genome-wide association analysis is a powerful tool for annotating phenotypic effects on the genome and knowledge of genes and chromosomal regions associated with dairy phenotypes is useful for genome and gene-based selection. Here, we report results of a genome-wide analysis of predicted transmitting ability (PTA) of 31 production, health, reproduction and body conformation traits in contemporary Holstein cows.</p> <p>Results</p> <p>Genome-wide association analysis identified a number of candidate genes and chromosome regions associated with 31 dairy traits in contemporary U.S. Holstein cows. Highly significant genes and chromosome regions include: BTA13's <it>GNAS </it>region for milk, fat and protein yields; BTA7's <it>INSR </it>region and BTAX's <it>LOC520057 </it>and <it>GRIA3 </it>for daughter pregnancy rate, somatic cell score and productive life; BTA2's <it>LRP1B </it>for somatic cell score; BTA14's <it>DGAT1-NIBP </it>region for fat percentage; <it>BTA1</it>'s <it>FKBP2 </it>for protein yields and percentage, BTA26's <it>MGMT </it>and BTA6's <it>PDGFRA </it>for protein percentage; BTA18's 53.9-58.7 Mb region for service-sire and daughter calving ease and service-sire stillbirth; BTA18's <it>PGLYRP1</it>-<it>IGFL1 </it>region for a large number of traits; BTA18's <it>LOC787057 </it>for service-sire stillbirth and daughter calving ease; BTA15's <it>CD82</it>, BTA23's <it>DST </it>and the <it>MOCS1</it>-<it>LRFN2 </it>region for daughter stillbirth; and BTAX's <it>LOC520057 </it>and <it>GRIA3 </it>for daughter pregnancy rate. For body conformation traits, BTA11, BTAX, BTA10, BTA5, and BTA26 had the largest concentrations of SNP effects, and <it>PHKA2 </it>of BTAX and <it>REN </it>of BTA16 had the most significant effects for body size traits. For body shape traits, BTAX, BTA19 and BTA3 were most significant. Udder traits were affected by BTA16, BTA22, BTAX, BTA2, BTA10, BTA11, BTA20, BTA22 and BTA25, teat traits were affected by BTA6, BTA7, BTA9, BTA16, BTA11, BTA26 and BTA17, and feet/legs traits were affected by BTA11, BTA13, BTA18, BTA20, and BTA26.</p> <p>Conclusions</p> <p>Genome-wide association analysis identified a number of genes and chromosome regions associated with 31 production, health, reproduction and body conformation traits in contemporary Holstein cows. The results provide useful information for annotating phenotypic effects on the dairy genome and for building consensus of dairy QTL effects.</p

Gender differences in patients with dizziness and unsteadiness regarding self-perceived disability, anxiety, depression, and its associations

BACKGROUND: It is known that anxiety and depression influence the level of disability experienced by persons with vertigo, dizziness or unsteadiness. Because higher prevalence rates of disabling dizziness have been found in women and some studies reported a higher level of psychiatric distress in female patients our primary aim was to explore whether women and men with vertigo, dizziness or unsteadiness differ regarding self-perceived disability, anxiety and depression. Secondly we planned to investigate the associations between disabling dizziness and anxiety and depression. METHOD: Patients were recruited from a tertiary centre for vertigo and balance disorders. Participants rated their global disability as mild, moderate or severe. They filled out the Dizziness Handicap Inventory and the two subscales of the Hospital Anxiety Depression Scale (HADS). The HADS was analysed 1) by calculating the median values, 2) by estimating the prevalence rates of abnormal anxiety/depression based on recommended cut-off criteria. Mann-Whitney U-tests, Chi-square statistics and odds ratios (OR) were calculated to compare the observations in both genders. Significance values were adjusted with respect to multiple comparisons. RESULTS: Two-hundred and two patients (124 women) mean age (standard deviation) of 49.7 (13.5) years participated. Both genders did not differ significantly in the mean level of self-perceived disability, anxiety, depression and symptom severity. There was a tendency of a higher prevalence of abnormal anxiety and depression in men (23.7%; 28.9%) compared to women (14.5%; 15.3%). Patients with abnormal depression felt themselves 2.75 (95% CI: 1.31-5.78) times more severely disabled by dizziness and unsteadiness than patients without depression. In men the OR was 8.2 (2.35-28.4). In women chi-square statistic was not significant. The ORs (95% CI) of abnormal anxiety and severe disability were 4.2 (1.9-8.9) in the whole sample, 8.7 (2.5-30.3) in men, and not significant in women. CONCLUSIONS: In men with vertigo, dizziness or unsteadiness emotional distress and its association with self-perceived disability should not be underestimated. Longitudinal surveys with specific pre-defined co-variables of self-perceived disability, anxiety and depression are needed to clarify the influence of gender on disability, anxiety and depression in patients with vertigo, dizziness or unsteadiness

Sulfite : Cytochrome c oxidoreductase from Thiobacillus novellus - Purification, characterization, and molecular biology of a heterodimeric member of the sulfite oxidase family

Direct oxidation of sulfite to sulfate occurs in various photo- and chemotrophic sulfur oxidizing microorganisms as the final step in the oxidation of reduced sulfur compounds and is catalyzed by sulfite:cytochrome c oxidoreductase (EC 1.8.2.1), Here we show that the enzyme from Thiobacillus novellus is a periplasmically located alpha beta heterodimer, consisting of a 40.6-kDa subunit containing a molybdenum cofactor and an 8.8-kDa monoheme cytochrome c(552) smbunit (midpoint redox potential, Em(8.0) = +280 mV), The organic component of the molybdenum cofactor was identified as molybdopterin contained in a 1:1 ratio to the Mo content of the enzyme. Electron paramagnetic resonance spectroscopy revealed the presence of a sulfite-inducible Mo(V) signal characteristic of sulfite:acceptor oxidoreductases. However, pH-dependent changes in the electron paramagnetic resonance signal were not detected. Kinetic studies showed that the enzyme exhibits a ping-pong mechanism involving two reactive sites. K-m values for sulfite and cytochrome c(550) were determined to be 27 and 4 mu M, respectively; the enzyme was found to be reversibly inhibited by sulfate and various buffer ions. The sorAB genes, which encode the enzyme, appear to form an operon, which is preceded by a putative extracytoplasmic function-type promoter and contains a hairpin loop termination structure downstream of sorB. While SorA exhibits significant similarities to known sequences of eukaryotic and bacterial sulfite:acceptor oxidoreductases, SorB does not appear to be closely related to any known c-type cytochromes